---
id: orientation
title: Orientation
sidebar_label: Orientation
---

import Tabs from '@theme/Tabs'
import TabItem from '@theme/TabItem'
import useBaseUrl from '@docusaurus/useBaseUrl'

<div class="image-container">
  <img width="283" src={useBaseUrl("img/example.png")} />
</div>

## Camera Sensor Orientation

A Camera sensor is configured to deliver buffers in a specific size, for example 4096x2160.
To avoid re-allocating such large buffers every time the phone rotates, the Camera pipeline will always deliver buffers in it's native sensor orientation (see [`sensorOrientation`](/docs/api/interfaces/CameraDevice#sensororientation)) and frames need to be rotated to appear up-right.

Since actually rotating pixels in such large buffers is really expensive and causes an unnecessary performance overhead, VisionCamera applies rotations through flags or transforms:

### Photo & Video Output

Photos and videos will be captured in a potentially "wrong" orientation, and VisionCamera will write an EXIF flag to the photo/video file with the correct presentation rotation.

:::info
This is handled automatically, and it's behaviour can be controlled via the [`outputOrientation`](/docs/api/interfaces/CameraProps#outputorientation) property.
:::

### Preview View

The Preview output will stream in a potentially "wrong" orientation, but uses view transforms (rotate + translate matrix) to properly display the Camera stream "up-right".

:::info
This will always happen automatically according to the screen's rotation.
:::

### Frame Processor Output

Frame Processors will stream frames in a potentially "wrong" orientation, and the client is responsible for properly interpreting the Frame data.

:::info
This needs to be handled manually, see [`Frame.orientation`](/docs/api/interfaces/Frame#orientation).
For example, in MLKit just pass the `Frame`'s `orientation` to the `detect(...)` method.

Instead of always rotating up-right to portrait, you might also want to rotate the Frame to either preview-, or output-orientation, depending on your use-case.
:::

## Implementation

VisionCamera supports three ways to implement orientation:

- Camera UI (preview view) is locked, but the buttons can rotate to the desired photo/video output orientation (**recommended**)
- Camera UI (preview view) also rotates alongside with the photo/video output orientation
- Both Camera UI (preview view) and photo/video output orientation are locked to a specific orientation

### The `outputOrientation` prop

The orientation in which photos and videos are captured can be adjusted via the [`outputOrientation`](/docs/api/interfaces/CameraProps#outputorientation) property:

```tsx
<Camera {...props} outputOrientation="device" />
```

- `"device"`: With the output orientation set to `device` (the default), photos and videos will be captured in the phone's physical orientation, even if the screen-rotation is locked.
This means, even though the preview view and other views don't rotate to landscape, holding the phone in landscape mode will still capture landscape photos. This is the same behaviour as in the iOS stock Camera.

- `"preview"`: Similar to `device`, the `preview` orientation mode will follow the phone's physical orientation, allowing the user to capture landscape photos and videos - but will always respect screen-rotation locks.
This means that the user is not able to capture landscape photos or videos if the preview view and other views stay in portrait mode (e.g. if the screen-lock is on).

### Listen to orientation changes

Whenever the output orientation changes, the [`onOutputOrientationChanged`](/docs/api/interfaces/CameraProps#onoutputorientationchanged) event will be called with the new output orientation. This is a good point to rotate the buttons to the desired output orientation now.

The [`onPreviewOrientationChanged`](/docs/api/interfaces/CameraProps#onprevieworientationchanged) event will be called whenever the preview orientation changes, which might be unrelated to the output orientation. Depending on the device's natural orientation (e.g. iPads being landscape by default), you should rotate all buttons on the UI relative to the preview orientation.

As a helper method, VisionCamera fires the [`onUIRotationChanged`](/docs/api/interfaces/CameraProps#onuirotationchanged) event whenever the target UI rotation changes. You can directly apply this rotation to any UI elements such as buttons to rotate them to the correct orientation:

```tsx
const [uiRotation, setUiRotation] = useState(0)
const uiStyle: ViewStyle = {
  transform: [{ rotate: `${uiRotation}deg` }]
}

return (
  <View>
    <Camera {...props} onUIRotationChanged={setUiRotation} />
    <FlipCameraButton style={uiStyle} />
  </View>
)
```

:::tip
For a smoother user experience, you should animate changes to the UI rotation. Use a library like react-native-reanimated to smoothly animate the `rotate` style.
:::

### The `Frame`'s `orientation`

In a Frame Processor, frames are streamed in their native sensor orientation.
This means even if the phone is rotated from portrait to landscape, the Frame's [`width`](/docs/api/interfaces/Frame#width) and [`height`](/docs/api/interfaces/Frame#height) stay the same.

The Frame's [`orientation`](/docs/api/interfaces/Frame#orientation) represents the **image buffer's orientation, relative to the device's native portrait mode**.

For example, if the Frame's `orientation` is `landscape-right`, it is 90° rotated and needs to be counter-rotated by -90° to appear "up-right".

On an iPhone, "up-right" means portrait mode (the home-button is at the bottom). On an iPad, "up-right" might mean a landscape orientation.

Instead of actually rotating pixels in the buffers, frame processor plugins just need to interpret the frame as being rotated.

MLKit handles this via a `orientation` property on the `MLImage`/`VisionImage` object:

```swift
public override func callback(_ frame: Frame, withArguments _: [AnyHashable: Any]?) -> Any? {
  let mlImage = MLImage(sampleBuffer: frame.buffer)
  // highlight-next-line
  mlImage.orientation = frame.orientation
  // ...
}
```

You can then either rotate to preview-, or output-orientation, depending on your use-case.

#### Rotate `Frame.orientation` to output Orientation

If you have a Frame Processor that detects objects or faces and the user holds the phone in a landscape orientation, your algorithm might not be able to detect the object or face because it is rotated.

In this case you can just rotate the `Frame.orientation` by the `outputOrientation` (see [`onOutputOrientationChanged`](/docs/api/interfaces/CameraProps#onoutputorientationchanged)), and it will then be a landscape Frame if the user rotates the phone to landscape, or a portrait Frame if the user holds the phone in portrait.

#### Rotate `Frame.orientation` to preview Orientation

If you have a Frame Processor tht applies some drawing operations or provides visual feedback to the Preview, you don't want to use the `outputOrientation` as that can be different than the `previewOrientation`.

In this case you can follow the same idea as above, just rotate the `Frame.orientation` by the `previewOrientation` (see [`onPreviewOrientationChanged`](/docs/api/interfaces/CameraProps#onprevieworientationchanged)) to receive a Frame in the same orientation the Preview view is currently in.

### Orientation in Skia Frame Processors

A Skia Frame Processor applies orientation via rotation and translation. This means the coordinate system stays the same, but output will be rotated accordingly.
For a `landscape-left` frame, `(0,0)` will not be top left, but rather top right.

## Mirroring (`isMirrored`)

The photo-, video- and snapshot- outputs can be mirrored alongside the vertical axis (left/right flipped) by setting [`isMirrored`](/docs/api/interfaces/CameraProps#ismirrored) to `true`.
By default, outputs are mirrored in the selfie camera, and not mirrored in any other cameras. To disable mirroring even for the selfie camera, just set `isMirrored` to `false`.

The preview view is always mirrored in the selfie camera, and never mirrored in any other cameras.

```tsx
<Camera {...props} isMirrored={true} />
```

<br />

#### 🚀 Next section: [Exposure](exposure)
